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Abstract. A three-dimensional virtual world simulation is described, where 
evolution takes place and it is possible to investigate behaviors of creatures in 
real-time. Bodies of these creatures are made of sticks, and their brains are built 
from artificial neurons. There are no constraints on topology and complexity of 
neural networks, as well as on the size of morphology. The model is inspired by 
biology, so energetic issues such as energy gains and losses are also considered. 
The evolutionary process can be guided by some pre-defined criteria, however, 
it is possible to mimic spontaneous evolution when the fitness is defined as the 
life span of the organisms. Interactions in the virtual world are discussed 
(including the possibility of worldwide-distributed simulation), and the results 
of so-far experiments are presented. 


1 Introduction 


Systems qualifying as virtual worlds have been built for a long time, but it is only 
recently that such systems achieved a superior level of realism, opening up the way 
for new applications. This is mainly due to the rapid development of computing 
machines, which now allow for fast simulation of complex systems. Such systems 
could earlier be investigated only theoretically. Another feature of present virtual 
world simulations is the existence of possible areas of their application, which range 
from medicine and economy to entertainment. 

Creating artificial worlds usually involves simulation of an environment and its 
habitants, interacting with each other and with the world they live in. Whereas such 
simulations of complex worlds are popular, it is not so easy to find the property of 
nature-like evolution included in the model [8, 14, 18]. Yet the evolution is an 
important source of complexity (emergence), because the amount of complexity 
explicitly designed in a system via its rules is limited. In particular, evolution is a 
force that can produce complexity going beyond the reach of the initial design. There 
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are many studies that focus on evolution of living forms within the field of Artificial 
Life [12]. However, these simulations are usually kept simple to focus on underlying 
mechanisms and allow for their efficient analysis and study [4, 5, 6]. 

The system described here tries to comprise the typical features of virtual world, 
together with its attractiveness for the human user, and to allow for evolution of the 
creatures which are simulated within the world. Although the simulation captures 
currently only a subset of properties of the natural world, its assumptions are inspired 
by natural laws [2, 17]. Moreover, the properties included are rich enough to allow a 
host of interactions to develop. Such a setup guarantees that the whole system is 
attractive for its observer/participant, and the evolutionary adaptation property 
together with the realistic three-dimensional simulation seem to be its most important 
features. 

While there have been experiments with the guided evolutionary processes in 
sophisticated systems [20], there was no attempt at spontaneous evolution using 
environments and creatures as complex as Framsticks. We hope that the presented 
model is suitable to give rise to sophisticated, real-like dependencies and behaviors. If 
they appear as the result of the spontaneous evolution, they will most probably 
astonish the beholder of Framsticks’ virtual world, thus displaying the emergence 
property [19]. 

In Framsticks, the three-dimensional world and creatures (their “bodies” and 
“brains”) are simulated. The creatures are capable of interacting with themselves 
(locating, pushing, hurting, killing, eating, etc.) and the environment (walking, 
swimming, ingesting, etc.). Framsticks are made of “sticks”, which allows for fast 
simulation, thus letting the evolution occur in a reasonable time. The virtual world 
itself can be composed of any combination of flat land, hills, and water. Despite 
simple simulation, this setup allows for realistic behaviors. 

Evolutionary process may be directed by some pre-defined criteria. One of them is 
life span, which can be used to mimic spontaneous evolution: the life span of a 
creature depends directly on energetic issues, and longer life span results in greater 
reproduction abilities. Various genetic representations are proposed, each of which is 
based on different qualitative ways of creature development. 

The paper is organized as follows: section 2 describes Framsticks system, its 
architecture, and models of simulation and evolution. Section 3 is devoted to the 
evolutionary properties of the system, describing various genotype representations, 
genetic operators, etc. Section 4 presents abilities of interaction within and with 
Framsticks simulator, and sections 5 and 6 discuss results of experiments, summarize 
the work and present future goals of the project. 


2 Simulation 


2.1 System architecture 


In general, our main goal is to design the model so that it allows for spontaneous 
(with natural selection), open-ended evolution. The creatures live in a three- 
dimensional world, and are controlled by neural networks. Currently, the system is 


more appropriate for guided evolutionary processes: the rules of selection and 
reproduction are pre-defined in the system architecture. That differs Framsticks from 
some other artificial life simulators (like Tierra [16] and Avida [1]), however, 
attempts have been made to avoid explicit reproduction rules [10]. 

In Framsticks system, the basic evolutionary framework is defined in a similar 
way, as in evolutionary models designed for optimization, such as genetic algorithms 
[7, 15] and most of artificial life simulations [3, 20, 21]. Therefore, evolution means 
optimization of creatures according to some predefined criteria. However, it is 
possible to mimic spontaneous evolution with the ‘directed’ model of evolution, when 
the fitness criterion is directly connected with the survival and reproduction abilities. 

The life span criterion is suitable for such a purpose. Using life span as fitness 
value affects selection phase: the longer a creature lives, the better it reproduces. One 
could simulate breeding of offspring or laying eggs, the amount of which would be 
proportional to the life span of the parent individuals, and would depend on their 
energy. However, there is no need to simulate those specific mechanisms as long as 
life span directly affects the reproductive success. Such an imitation of spontaneous 
evolution is reasonable. 

Simulating creatures in the virtual world is needed to estimate their fitness. In our 
model, it is possible to adjust the number n of individuals which are simulated at the 
same time. Thus, the virtual 
world does not contain the whole 
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In order to construct such an architecture (see fig. 1), two parameters are needed: 
the maximum total number of genotypes, N, and the maximum number of individuals 
simultaneously simulated, n. Individuals are selected to be simulated and evaluated 
when there is a free place in the virtual world (when the number of simulated 
creatures is less than n). When an individual dies, the performance of its genotype is 
updated. 

Usually, n is significantly smaller than N. When the interaction between simulated 
creatures is not important, n may be set to 1 (only one organism simulated at a time). 


Fig. 1. Architecture of the system. 


Larger values of n mean that a larger part of the whole set of individuals is simulated, 
and more interactions between creatures may happen. Thus, when setting the value 
for n, the size of the virtual world and the goal of evolution should also be considered. 

These rules can also be employed when the simulation is distributed. In such a 
case, it is assumed that the virtual world is not divided into areas or individuals, which 
would be then simulated in parallel. What is divided is the main gene pool, which is 
sent in parts to computing sites for evolution, and after some time these sites send 
their gene pools back to the main process. 


2.2 Physical simulation 


Framsticks simulates a three-dimensional world and creatures (finite element method 
is used for step-by-step simulation). We decided to use such a sophisticated 
environment for evolution hoping that a range of complex, various stimuli affecting 
organisms will be the origin of dynamic development and interesting behaviors will 
emerge. The environment is used to evaluate individuals (genotypes), and the first 
behaviors tested were the mechanisms of locomotion and orientation. We considered 
all the kinds of interaction between physical objects: static and dynamic friction, 
damping, action and reaction forces, energy losses after deformations, gravitation, and 
uplift pressure — buoyancy (in water environment). The most important forces are 
shown on figure 2. 


elastic 
reaction 
of joints 


ground 
reaction 


Fig. 2. Forces involved in the simulation. 


One should note that there is always a tradeoff between simulation accuracy and 
simulation time. We need fast simulation to perform evolution, on the other hand, the 
system should be as realistic as possible to produce realistic behaviors. 

Currently, in order to make the simulation faster and due to the computational 
complexity, some aspects were discarded: collisions between parts of an organism 
itself and the movement of a water medium were both ignored. Including these in the 
simulation would make it more realistic, but would not introduce any qualitatively 
new (important from the viewpoint of the simulated creatures) phenomena. From the 
viewpoint of evolution and the investigator of the evolved behaviors, meaningful 
interactions are more important than very realistic, but too slow simulation [13]. 


The current model implies some constraints concerning structure of organisms. 
The basic element is a stick made of two flexibly joined particles (finite element 
method is used for simulation). Sticks have specific properties: biological (muscle 
strength, stamina, energetic: assimilation, ingestion, and initial energy level), physical 
(length, weight, friction), and concerning stick joints (rotation, twist, curvedness). 

Neurons (connected in any way) and receptors can also be placed on sticks. Rather 
than evolving neural networks exclusively [3], morphology — “body” is evolved 
together with “brain” in Framsticks. 

Muscles are placed on stick joints. There are two kinds of muscles: bending and 
rotating. When a muscle is sent a zero signal, it becomes neutral (as if it was absent). 
Positive and negative changes make the sticks move in either direction — it is 
analogous to natural systems of muscles, with flexors and extensors. The strength of a 
muscle determines its effective ability of movement and speed (acceleration). 
However, stronger muscle consumes more energy during its work. Additionally, an 
increase in the “strength” property of a muscle results in a decrease of the other 
properties of a stick. 

Framsticks have currently three kinds of receptors (senses): those for orientation in 
space (equilibrium sense, gyroscope), detection of energy/food (non-directional 
smell) and detection of physical contact (directional touch). Receptors and effectors 
are shown on figure 3. Details are illustrated at [11]. 
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Fig. 3. Senses and effectors of Framsticks. 


2.3 Neural network 


In order to enrich capabilities of creatures’ brains, the typical artificial neurons were 
equipped with three additional parameters which can alter the behavior of each 
neuron independently. As Nature was our inspiration, we did not introduce 
sophisticated and unnatural processing units (as in [20]); it is possible to construct 
complex modules (integrating, differentiating, summing, subtracting, and generators 
with different shapes) from simple neurons. 

The additional neuron parameters (force, inertia and sigmoid) are under control of 
evolution. They affect the way neurons work: Force and inertia influence changes of 
the inner neuron state (speed and tendency of changes, respectively). Sigmoid 


coefficient modifies the output function. Details and sample neuronal runs can be 
found at [11]. 

The neural network can have any topology and complexity. Neurons can be 
connected with each other in any way (some may be unconnected). Inputs can be 
connected to outputs of another neurons, constant values, or senses, while outputs can 
be connected to inputs of another neurons or effectors (muscles). 


3 Evolution 


3.1 Genotype representations 


In order to test the properties and abilities of evolutionary processes, various genotype 
encodings have been designed. All of them are expressed with symbols, so they can 
be relatively easily investigated, modified and understood. All of them combine 
“body” and “brain” in the same genotype, so that both morphology (body made of 
sticks) and control (brain made of neurons) evolve at the same time. 

The common basic layer for describing a phenotype is the basic direct encoding. It 
is least restrictive and allows for building of any creatures which can be simulated 
within the virtual world. Such a genotype simply lists all the parts of an organism. 

The recurrent direct encoding was designed to allow human users to build their 
own creatures in a possibly simple way, nevertheless some important properties of the 
encoding were also maintained to make the evolution efficient. Generally, users can 
write recurrent genotype expressions, which are interpreted as phenotypes. Due to the 
recurrent nature of the language, the morphology has a tree-like structure. Special 
signs can be used to modify (increase or decrease) properties of parts of the body (see 
2.2). “Brain” is made of neurons with their connections described relatively. Two 
genetic operators are used: crossover and mutation. It often happens that genotypes 
are invalid after crossover and mutation because of small errors. That is why a simple 
repair procedure is used. This procedure can fix little mistakes (like references to non- 
existing neurons) and validate an invalid genotype. 

In nature, the genetic code of complex organisms does not encode their body 
layout directly, but rather their process of development. That is why a developmental 
encoding was designed. An interesting merit of developmental encoding is that it can 
incorporate symmetry and modularity, features commonly found in natural systems. 
The developmental encoding is similar to the recurrent direct one, but codes are 
interpreted as commands by cells (sticks, neurons, etc.). Cells can change their 
parameters, and divide. 

Another encodings (like direct similarity development and implicit embryogeny 
development) are also proposed in order to test various representations of creatures 
(morphologies and brains), and the influence of the representations on the 
evolutionary process. Trying various approaches to describe creatures is also 
important from the viewpoint of a human user: some representations may be better 
than others to use in a graphical, user-friendly editor of phenotypes. It should be as 
easy as possible to build a creature for non-experts, on the other hand, they should be 


able to control details of such design if needed. It is relatively easy to add support for 
more encodings suggested by developers. 


3.2 Selection and speciation 


When a new individual has to be created, the simulator selects an existing genotype 
using the standard roulette wheel rule [7], proportionally to its fitness. The genetic 
operators are then applied. 

In nature, groups of similar individuals share the same ecological niche and 
constitute one species. In Framsticks, similarity to other coexisting species may lower 
the given species’ fitness [7]. This introduces a pressure for populations of species to 
diversify. The second mechanism which allows speciation is the specific crossover 
operation: if possible, corresponding parts of genotypes of similar species are 
exchanged. 

The described mechanism of aiding speciation by lowering fitness of similar 
genotypes is somehow artificial However, such a phenomenon happens 
spontaneously in Framsticks when a large number of individuals are simulated at the 
same time in the virtual world. Then, depending on the behavior, if many individuals 
exhibit the same method of achieving energy, their situation will become more 
difficult, and competition within species will arise. 


3.3 Creatures’ evaluation 


During the life span of simulated organisms some features are measured, like distance 
(center of gravity movements), age, average velocity (equal to distance/age) etc. In 
practice, movements are not measured from the moment of birth, but from the 
moment when a creature becomes stable. This is because some oscillations take place 
immediately after creation, causing shifts of the center of gravity. Measuring those 
movements would be unfair, as they are not caused by the creature itself, and do not 
depend on a creature’s ability to move on its own. 

After a creature dies, all the values of various performance criteria are weighted to 
produce the final fitness value for the given genotype. 


4 Interaction 


Interaction happens on many levels of the Framsticks system architecture. There is 
interaction between creatures and their world, where senses are the sources of 
information for organisms, and muscles are their way to influence their own state and 
their environment. 

Potentially, as there is no limitation concerning the complexity of neural network, 
creatures can become intelligent, and it is unknown to what degree. Framsticks can 
learn about their orientation, and sense their world (and, potentially, remember it). 
From the set of three senses, the smell receptor is the most important. It allows to find 
food, but also to tell food from other creatures. With the proper use of many smell 


sensors, creatures can virtually discriminate between various morphologies of other 
habitants, sense their movement, their number, etc. The great help of complex brain 
allows for the unlimited use and exploitation of smell, even for diversification of 
species, group and social behaviors, and mutual preferences. If the simulator is very 
complex, it may resemble The Matrix [22] from the viewpoint of the creatures, and 
they can learn its imperfections in order to utilize them (as it was already observed, 
but it was evolution what explored simulator faults). 

In Framsticks, there is a lot of the interaction with the human role. It is the human 
who can affect living creatures, revive them (put into artificial world), kill (remove 
from the world), move in the world with the “robotic arm”, feed them (put energy into 
the world), cause crashes etc. As Framsticks interface may show simulation in real- 
time, responses to user actions are instantaneous and highly visual. This facilitates the 
exploration of many aspects of evolutionary process, like fitness criteria, creation of 
offspring (mutation, crossover, cloning, etc.), population mixing, partial evolution, 
etc. Through the process of interactive experimentation, users can develop an 
understanding of some of the fundamentals of evolutionary dynamics. Users can also 
affect the gene pool by designing their own creatures, editing genotypes, and 
modifying and improving them with an instant preview of phenotypes. 

It is possible to define the virtual world: its size, hills, slopes, water level, and 
boundaries (like walls, teleport, or no boundaries). Humans can adjust the architecture 
for the evolutionary process (maximum numbers for genotypes and simulated 
individuals), turn on or off destructive collisions (whether collisions break creatures), 
adjust the rate of auto-feeding, adjust energy usage rates for brain and body work, etc. 

As the evolutionary process is the source of development in the Framsticks world, 
the most powerful ability is to understand and adjust the parameters of evolution, like 
weights for fitness criteria, origin of offspring (mutation, crossover, cloning, etc.), 
mutation rates, speciation, the ability to exclude some creatures’ elements from 
evolution, etc. 

Finally, the virtual world itself affects a human user, who is influenced by what 
they can see, the way creatures behave: move, seek energy, swim, walk, fight, etc. 
This kind of interaction is the subtlest one, as it does not concern the apparent action, 
but it certainly happens. When the simulation is distributed over Internet [11], the 
collective, worldwide experiments will connect their participants, and enable sharing 
of ideas, knowledge and experience of the system, analysis of results, and discussions. 
Distributing evolution of realistic 3D creatures in large scale may change the way 
people think about evolution, making it more (or less) unusual and ambiguous. 
Eventually, experiments with open-ended setups may bring unexpected results. 


5 Results 


The majority of our experiments concerned directed evolution, with fitness defined as 
speed (on the ground or in water). Many walking and swimming species evolved 
during these runs [11], and we were able to see the evolution of ideas of “how to 
move” [9]. In one evolutionary run, a limb was doubled while crossing over, and after 
some further evolution the organism was able to move with two limbs — one for 


pushing back and one for pulling [11]. We also noticed a case when a limb is 
simultaneously bent and rotated, which was a more effective method of pushing 
against the ground. In one case a neuron used its saturation to produce delayed 
signals, which is a kind of a simple “short-term memory’. More sophisticated 
creatures which were evolved could not be easily examined because of their high 
complexity. 

Generally, as long as there is no really good way of locomotion in a population, 
there is a tendency to try a lot of ideas. The creatures with stronger muscles, better 
control and many limbs (to achieve stability) survive. Evolution tends to promote 
strength, and make the pushing/pulling limbs longer and rougher. The really great 
innovations seem to be discovered once and then sustained throughout the run, until 
they are replaced by better ideas. We noticed that it was hard to evolve a low- 
frequency signal generator, perhaps because it cannot be created by little, step-by-step 
improvements. Movement in water was easier to achieve than on land, see figure 4 for 
example of swimming movement (evolutionary redundancy and random character can 
be observed). 
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The experiments concerning evolutionary improvement of human pre-designed 
structures have also been conducted. An example is the evolution of control (neural 
network) for a pre-designed morphology in order to obtain a creature that lives long. 
In order to live long, it is required to find energy sources and ingest them. The user 
can design a genotype, and then turn off crossover and structure mutations, allowing 
only mutations of neural network (connections, topology and weights). Thus a 


food-seeking creatures are evolved on the basis of pre-designed morphology. See 
figure 5 for an example of pre-designed body and brain, with further directed 
evolution of movement. 
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Fig. 5. Pre-designed creature (“lizard”). Neural network connections and weighs evolved with 
fitness defined as speed. One touch and one equilibrium receptors. Walks in a realistic way. 


6 Discussion and future work 


During our evolutionary experiments, we had to modify simulation rules and fix bugs 
several times. Evolution turned out to be a very good method of searching the space 
of solutions (organisms), and was capable of finding good individuals, regardless of 
their sensibility and validity. Faults in the simulator were sooner or later discovered 
by evolution and exploited to the highest possible extent in simulated organisms. 

There are a few limitations of the current model. The genotypic representations 
need to be investigated and tuned to allow efficient evolution of creatures, not 
restricting various, complex body and brain plans. Our future work will thus concern 
various genotype representations (see 3.1), implementation of specific genetic 
operators, experiments and their study. We hope that it will make the search of the 
organism space even more effective. The preliminary results show that it will lead to 
creation of more diverse constructions and their easier modification. A suitable 
representation for user-friendly design of creatures will also be investigated. 

Our future work will also concern defining a similarity function on the phenotype 
space, improvements of simulation rules and their parameters, and spontaneous, open- 
ended simulations. More receptors and more complex criteria for directed evolutions 


may be introduced. If the increase in computing power permits, we will use more 
realistic (but slower) physical simulator, so that the evolved creatures will behave 
more like if they would be built in the real world. Generally, realism and open- 
endedness are properties that require much more computation and effort than simple 
and/or directed evolutionary models, but we hope to employ Internet worldwide 
distribution to fully exploit capabilities of the system. 

The results we have achieved are promising, and it is really interesting to look 
forward to the behaviors and phenomena that will emerge with the new genotype 
encodings, and after collective, distributed experiments are performed. 
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